feat(logger): align sampling debug logs feature implementation with the other runtimes #1744

shdq · 2023-10-13T23:28:05Z

Description of your changes

This PR introduces a new implementation for the sampling debug logs feature. It aligns with other runtimes, where log sampling decision is done at the per-function level.

Main changes:

logsSampled field was removed in favor of the sample_rate
setInitialSampleRate method initializes sampleRateValue in order like in initial log level method:

constructor
custom config
env var
default value (zero), it's never undefined now.

validation added to sampleRateValue to be in range from 0.0 to 1.0. Before it could be any number. It doesn't throw an error and silently ignores invalid values. This could be improved on the types level, to only allow values in this range, but it's not trivial.

Related issues, RFCs

Issue number: #1136

Checklist

My changes meet the tenets criteria
I have performed a self-review of my own code
I have commented my code where necessary, particularly in areas that should be flagged with a TODO, or hard-to-understand areas
I have made corresponding changes to the documentation
My changes generate no new warnings
I have added tests that prove my change is effective and works
The PR title follows the conventional commit semantics

Breaking change checklist

Is it a breaking change?: NO

I have documented the migration process
I have added, implemented necessary warnings (if it can live side by side)

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Disclaimer: We value your time and bandwidth. As such, any pull requests created on non-triaged issues might not be successful.

…function level

packages/logger/src/Logger.ts

dreamorosi · 2023-10-14T00:52:52Z

Hey Sergei, thanks a lot for the PR!

I need a bit more time to review it and try the code, but at first look it seems like you are in going in the right direction 😄

To answer your questions:

I removed refreshSampleRateCalculation() for now, should we make it an alias to configureSampling()?

In Python the method seems to be exclusively private (it starts with _), so maybe we don't need to.

Should we pollute log output when the log level is changed to DEBUG because of the sampling rate?
this.debug('Setting log level to DEBUG due to sampling rate') This behavior is ported from Python.

I'm not sure yet to be honest. On one hand, I can see how this could be useful to let people know that the log level has changed because of this feature.

On the other hand, I do kind of agree with your choice of wording (polluting), so I'm not sure.

Let me think about it a bit longer and I'll give you an answer when I review more in depth and run the code a few times, either way it should be an easy addition.

…t of the method importance

shdq · 2023-10-15T17:57:48Z

I removed refreshSampleRateCalculation() for now, should we make it an alias to configureSampling()?

In Python the method seems to be exclusively private (it starts with _), so maybe we don't need to.

My concern is that refreshSampleRateCalculation() is exposed in docs, so it would be another breaking change if we remove it.

…r sampling rate feature, improve description

dreamorosi · 2023-10-18T17:51:57Z

My concern is that refreshSampleRateCalculation() is exposed in docs, so it would be another breaking change if we remove it.

You are right, I would say let's keep it for now and adopt your suggestion.

packages/logger/src/Logger.ts

dreamorosi

This is looking really good!

I have pulled the repo and made some suggestions. Also, as discussed on Discord, I'm sharing the modifications to adapt the integration tests.

In the packages/logger/tests/e2e/sampleRate.decorator.test.ts file, in the test case that starts with describe('Enabling sample rate', () => { (around L70), change the if/else to this:

if (logMessages.length === 1 && logMessages[0].includes('ERROR')) {
  countNotSampled++;
} else if (
  logMessages.length === 5 &&
  logMessages[0].includes(
    'Setting log level to DEBUG due to sampling rate'
  )
) {
  countSampled++;
} else {
  console.error(`Log group ${logGroupName} contains missing log`);
  throw new Error(
    'Sampled log should have either 1 error log or 4 logs of all levels'
  );
}

As you can see the test is the same, but now we are looking for 5 log entries when log sampling is enabled. The new log entry is the one that Logger emits to notify the user that debug log sampling is enabled, so we assert that it contains the correct string.

In order for this to work, we need to do a slight modification to the TestInvocationLogs class, which is used in this test (invocationLogs[i].getFunctionLogs()). The modification is needed because up until now we were looking only at log entries between the START and END logs of the invocation (which are the logs emitted within the scope of the function handler).

Now however we need also look at the ones emitted during the initialisation phase, since the debug log (Setting log level to DEBUG due to sampling rate) is emitted outside of the handler.

To do this we need to modify the file packages/testing/src/TestInvocationLogs.ts to the content below (collapsed for brevity).

import { LogLevel } from './constants.js';
import type { FunctionLog } from './types.js';

const CloudWatchLogKeywords = {
  END: 'END RequestId',
  INIT_START: 'INIT_START',
  REPORT: 'REPORT RequestId',
  START: 'START RequestId',
  XRAY: 'XRAY TraceId',
} as const;

class TestInvocationLogs {
  public static LEVEL = LogLevel;

  /**
   * Array of logs from invocation.
   *
   * The first element is START, and the last two elements are END, and REPORT.
   *    [
   *      'START RequestId: c6af9ac6-7b61-11e6-9a41-93e812345678 Version: $LATEST',
   *      '{"cold_start":true,"function_arn":"arn:aws:lambda:eu-west-1:561912387782:function:loggerMiddyStandardFeatures-c555a2ec-1121-4586-9c04-185ab36ea34c","function_memory_size":128,"function_name":"loggerMiddyStandardFeatures-c555a2ec-1121-4586-9c04-185ab36ea34c","function_request_id":"7f586697-238a-4c3b-9250-a5f057c1119c","level":"INFO","message":"This is an INFO log with some context and persistent key","service":"logger-e2e-testing","timestamp":"2022-01-27T16:04:39.323Z","persistentKey":"works"}',
   *      '{"cold_start":true,"function_arn":"arn:aws:lambda:eu-west-1:561912387782:function:loggerMiddyStandardFeatures-c555a2ec-1121-4586-9c04-185ab36ea34c","function_memory_size":128,"function_name":"loggerMiddyStandardFeatures-c555a2ec-1121-4586-9c04-185ab36ea34c","function_request_id":"7f586697-238a-4c3b-9250-a5f057c1119c","level":"INFO","message":"This is an INFO log with some context","service":"logger-e2e-testing","timestamp":"2022-01-27T16:04:39.323Z","persistentKey":"works","additionalKey":"additionalValue"}',
   *      '{"cold_start":true,"function_arn":"arn:aws:lambda:eu-west-1:561912387782:function:loggerMiddyStandardFeatures-c555a2ec-1121-4586-9c04-185ab36ea34c","function_memory_size":128,"function_name":"loggerMiddyStandardFeatures-c555a2ec-1121-4586-9c04-185ab36ea34c","function_request_id":"7f586697-238a-4c3b-9250-a5f057c1119c","level":"ERROR","message":"There was an error","service":"logger-e2e-testing","timestamp":"2022-01-27T16:04:39.323Z","persistentKey":"works","error":{"name":"Error","location":"/var/task/index.js:2778","message":"you cannot prevent this","stack":"Error: you cannot prevent this\\n    at testFunction (/var/task/index.js:2778:11)\\n    at runRequest (/var/task/index.js:2314:36)"}}',
   *      'END RequestId: c6af9ac6-7b61-11e6-9a41-93e812345678',
   *      'REPORT RequestId: c6af9ac6-7b61-11e6-9a41-93e812345678\tDuration: 2.16 ms\tBilled Duration: 3 ms\tMemory Size: 128 MB\tMax Memory Used: 57 MB\t',
   *    ]
   * See https://docs.aws.amazon.com/lambda/latest/dg/nodejs-logging.html for details
   */
  private logs: string[];

  public constructor(logResult: string) {
    const rawLog = Buffer.from(logResult, 'base64').toString('utf-8').trim();
    this.logs = rawLog.split('\n');
  }

  /**
   * Find all functional logs whether it contains a given text
   * @param text
   * @param levelToFilter level to filter
   * @returns
   */
  public doesAnyFunctionLogsContains(
    text: string,
    levelToFilter?: keyof typeof LogLevel
  ): boolean {
    const filteredLogs = this.getFunctionLogs(levelToFilter).filter((log) =>
      log.includes(text)
    );

    return filteredLogs.length > 0;
  }

  /**
   * Return all the log of the function
   * @returns Array of function logs
   */
  public getAllFunctionLogs(): string[] {
    return this.logs;
  }

  /**
   * Return the index of the log that contains `END RequestId`
   * @param logs
   * @returns {number} index of the log that contains END RequestId
   */
  public static getEndLogIndex(logs: string[]): number {
    return logs.findIndex((log) => log.startsWith(CloudWatchLogKeywords.END));
  }

  /**
   * Return only logs from function, exclude INIT_START, START, END, REPORT,
   * and X-Ray log generated by the Lambda service.
   *
   * @param {typeof LogLevel} [levelToFilter] - Level to filter the logs
   * @returns Array of function logs, filtered by level if provided
   */
  public getFunctionLogs(levelToFilter?: keyof typeof LogLevel): string[] {
    const exclusionKeywords = Object.values(CloudWatchLogKeywords);
    let filteredLogs = this.logs.filter(
      (log) => !exclusionKeywords.some((keyword) => log.startsWith(keyword))
    );

    if (levelToFilter) {
      filteredLogs = filteredLogs.filter((log) => {
        try {
          const parsedLog = TestInvocationLogs.parseFunctionLog(log);

          return parsedLog.level == levelToFilter;
        } catch (error) {
          // If log is not from structured logging : such as metrics one.
          return (
            (log.split('\t')[2] as keyof typeof LogLevel) === levelToFilter
          );
        }
      });
    }

    return filteredLogs;
  }

  /**
   * Return the index of the log that contains `INIT_START`
   * @param logs
   * @returns {number} index of the log that contains `INIT_START`
   */
  public static getInitLogIndex(logs: string[]): number {
    return logs.findIndex((log) =>
      log.startsWith(CloudWatchLogKeywords.INIT_START)
    );
  }

  /**
   * Return the index of the log that contains `REPORT RequestId`
   * @param logs
   * @returns {number} index of the log that contains `REPORT RequestId`
   */
  public static getReportLogIndex(logs: string[]): number {
    return logs.findIndex((log) =>
      log.startsWith(CloudWatchLogKeywords.REPORT)
    );
  }

  /**
   * Return the index of the log that contains `START RequestId`
   * @param logs
   * @returns {number} index of the log that contains `START RequestId`
   */
  public static getStartLogIndex(logs: string[]): number {
    return logs.findIndex((log) => log.startsWith(CloudWatchLogKeywords.START));
  }

  /**
   * Return the index of the log that contains `XRAY TraceId`
   * @param logs
   * @returns {number} index of the log that contains `XRAY TraceId`
   */
  public static getXRayLogIndex(logs: string[]): number {
    return logs.findIndex((log) => log.startsWith(CloudWatchLogKeywords.XRAY));
  }

  /**
   * Each of log message contains a JSON with the structured Log object (e.g. {\"cold_start\":true, ..})
   * @param log
   */
  public static parseFunctionLog(log: string): FunctionLog {
    return JSON.parse(log);
  }
}

export { TestInvocationLogs };

After making the change in the testing package, if you want to run the tests, you'll have to run npm run build -w pacakges/testing first.

Hope this helps!

sonarqubecloud · 2023-10-19T21:45:57Z

Kudos, SonarCloud Quality Gate passed!

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
20.8% Duplication

shdq · 2023-10-19T21:51:45Z

Oh, one last thing to update is the docs section.

dreamorosi

I have just run the integration tests and they're passing 🎉

As discussed on Discord, I'm going to take care of the docs in a separate PR since I have to also create the v2 update guide.

Thanks A TON for the work on this, it's highly appreciated!

dreamorosi · 2023-10-19T22:50:12Z

The CI checks are failing due to an extra comma that I introduced a while ago in a previous commit - this is due to the version of Prettier on main being older than the one in this branch and it's formatting in that specific case having changed.

I have already fixed the issue in another commit on the branch being merged into (here), and the file wasn't touched in this PR, so merging anyway.

…he other runtimes (#1744) * test(logger): remove logsSampled field, add default sampleRateValue * test(logger): add tests for sampling debug logs feature * feat(logger): change implementation to make sampling decision at per-function level * refactor(logger): remove redundant createLogger method * refactor(logger): remove getSampleRateValue method * test(logger): improve tests * refactor(logger): return createLogger() back with the detailed comment of the method importance * test(logger): add constructor/custom config/env var priority tests for sampling rate feature, improve description * refactor(logger): address review comments * feat(logger): add refreshSampleRateCalculation method and tests * test(logger): adjust end-to-end tests

* feat(logger): Support for external observability providers (#1511) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter --------- Co-authored-by: erikayao93 <[email protected]> * chore(logger): PowertoolsLogFormatter docstring and variable naming update (#1585) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter * Fixed incorrect return type, renamed variable for consistency * feat(logger): Support for external observability providers (#1511) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter --------- Co-authored-by: erikayao93 <[email protected]> * chore(logger): PowertoolsLogFormatter docstring and variable naming update (#1585) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter * Fixed incorrect return type, renamed variable for consistency * chore(maintenance): bump dependencies & drop nodejs14x (#1687) * chore: update release script to mark all utilities as alpha * chore: restore version to ease conflicts * chore: release version change * chore: release version change * chore(maintenance): remove `createLogger` and `createTracer` helpers (#1722) * chore(maintenance): bump dependencies & drop nodejs14x (#1687) * chore: add pre-release script * chore: restore deps * chore: added v2 shim * chore(maintenance): remove logger and tracer helper function * chore: remove imports * chore: fix deps & versions * tests: moved unit tests * tests: move logger tests * chore: added v2 shim * chore: added v2 shim * feat(logger): add esmodule support (#1734) * feat(logger): add esm build output * fix(Logger): Remove barrel files update references * test(Logger): update jest/ts-jest to use ESM * chore(Logger): remove unused lodash.merge * fix(logger): reinstate lodash.merge * chore(logger): revert TS assertion * chore(logger): revert format changes * chore(logger): update postbuild to remove incremental tsbuildinfo files * fix(logger): correct reference to types output * feat(logging): add middleware export * chore(logger): replace postbuild script with echo statement * feat(logger): add typesVersions property and barrel files to /types * chore(logger): file not used, can be added back if needed * chore(logger): add space back to README * chore(logger): revert space in README * feat(commons): add esmodule support (#1735) * chore(logger): adapt logger to commons exports * feat(commons): add esmodule support * chore: address sonar findings * chore(commons): exported version * chore: fixed imports in examples * chore(parameters): fixed imports * chore(metrics): fixed imports * chore(tracer): fixed imports * chore(idempotency): fixed imports * chore(commons): test coverage * chore(batch): fix imports * feat(parameters): add esmodule support (#1736) * feat(batch): add esmodule support (#1737) * feat(internal): add esmodule support (#1738) * feat(testing): add esmodule support * chore(all): update imports * feat(metrics): add esmodule support (#1739) * feat(tracer): add esmodule support (#1741) * feat(tracer): add esmodule support * chore(docs): update imports * feat(idempotency): add esmodule support (#1743) * feat(idempotency): add esmodule support * chore(metrics): fix import * chore(ci): v2 release line * chore(ci): fix alpha versioning pre-release * docs(maintenance): add processes tab (#1747) * docs(maintenance): update mkdocs to support tabs * chore(ci): add parallel test npm script * chore(ci): add jest command * docs(maintenance): add testing page to navbar * docs(maintenance): add contributing info * chore: update roadmap * chore: update release drafter workflow to allow for manual trigger * fix formatting * docs: maintainers handbook * chore: link to new location * fix links * Update docs/maintainers.md Co-authored-by: Alexander Schueren <[email protected]> --------- Co-authored-by: Alexander Schueren <[email protected]> * chore(docs): add invisible unicode char to decorator docs (#1755) * chore: remove extra comma * chore(docs): upgrade doc intro * feat(logger): align sampling debug logs feature implementation with the other runtimes (#1744) * test(logger): remove logsSampled field, add default sampleRateValue * test(logger): add tests for sampling debug logs feature * feat(logger): change implementation to make sampling decision at per-function level * refactor(logger): remove redundant createLogger method * refactor(logger): remove getSampleRateValue method * test(logger): improve tests * refactor(logger): return createLogger() back with the detailed comment of the method importance * test(logger): add constructor/custom config/env var priority tests for sampling rate feature, improve description * refactor(logger): address review comments * feat(logger): add refreshSampleRateCalculation method and tests * test(logger): adjust end-to-end tests * chore(logger): refactor types and interfaces (#1758) * chore(logger): refactor types and interfaces * chore: grouped type files * chore: fix code smell * chore: fix ci * chore: fix ci * chore(maintenance): bump Middy v4 & run tests (#1760) * chore(parameters): fix esm bundling * chore(parameters): change declare client param * chore(layers) widen version check in e2e * chore(maintenance): enable `isolatedModules` and isolate cache (#1765) * chore(layers) widen version check in e2e * chore(maintenance): enable isolatedModules * chore: remove redundant comments from tsconfig * chore: changed path of tsbuild cache * fix: idempotency types * build(maintenance): bump aws sdk dev dependencies * chore(parameters): add export types * chore(logger): set default UTC timezone (#1775) * chore(parameters): add export types * chore(logger): set default utc timezone * chore(logger): pass down envvarsservice to log formatter * fix(metrics): deduplicate dimensions when serialising (#1780) * fix: deduplicate dimensions when serializing * fix tests * remove tsbuildinfo * remove whitespace * fix gitignore again * play some sonar games * fix test * chore(release): v1.14.2 [skip ci] * feat(metrics): log directly to stdout (#1786) * chore(commons): move isDevMode to commons * chore(logger): move isDev config out of logger to commons * feat(metrics): use own console object by default * tests(layers): fix unit tests * feat(maintenance): add support for nodejs20.x runtime (#1790) * feat(maintenance): support nodejs20.x runtime * tests(metrics): fix object ordering in tests * build(testing): bump cdk * build(maintenance): revert aws-cdk-lib to support nodejs14 * tests(maintenance): set runtime with family * chore(docs): patch runtime in cdk * chore(docs): patch runtime in cdk * chore(maintenance): increment version in commons ahead of release * chore(commons): simplify config service interface * chore(release): v1.15.0 [skip ci] * feat(logger): add support for `AWS_LAMBDA_LOG_LEVEL` and `POWERTOOLS_LOG_LEVEL` (#1795) * feat(logger): support advanced logging * docs(logger): add alc info * feat(logger): support alc * docs: fix alc docs links * tests(logger): add unit tests for the feature * docs(logger): make POWERTOOLS_LOG_LEVEL default * chore(release): v1.16.0 [skip ci] * chore(maintenance): drop support for Node.js 14 (#1802) * chore: package lock * chore(docs): update docs url in comments & readme files (#1728) * chore(ci): update v2 release workflow (#1745) * docs(maintenance): add processes tab (#1747) * docs(maintenance): update mkdocs to support tabs * chore(ci): add parallel test npm script * chore(ci): add jest command * docs(maintenance): add testing page to navbar * docs(maintenance): add contributing info * chore: update roadmap * chore: update release drafter workflow to allow for manual trigger * fix formatting * docs: maintainers handbook * chore: link to new location * fix links * Update docs/maintainers.md Co-authored-by: Alexander Schueren <[email protected]> --------- Co-authored-by: Alexander Schueren <[email protected]> * chore(maintenance): add --require-hashes flag to pip installs (#1827) * bump version to 9.5.2, rerun pip-compile with correct deps (#1830) * chore(ci): Dependabot fine tuning (#1862) * ignore major updates for mike * set versioning strategy for cdk, dependabot is running on auto upgrade strategy per default * ignore middy major upgrades * remove CodeQL, it's enough to have Sonar * update package-lock * chore(deps-dev): bump @aws-sdk/client-cloudwatch from 3.438.0 to 3.485.0 (#1857) * chore(deps-dev): bump @types/node from 20.11.0 to 20.11.2 (#1912) * chore(deps): fix dependencies and dependabot config (#1917) * chore(deps-dev): bump @typescript-eslint/parser from 6.19.0 to 6.19.1 (#1946) Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 6.19.0 to 6.19.1. - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/parser/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v6.19.1/packages/parser) --- updated-dependencies: - dependency-name: "@typescript-eslint/parser" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(deps-dev): bump @typescript-eslint/eslint-plugin (#1948) Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 6.19.0 to 6.19.1. - [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases) - [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/main/packages/eslint-plugin/CHANGELOG.md) - [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v6.19.1/packages/eslint-plugin) --- updated-dependencies: - dependency-name: "@typescript-eslint/eslint-plugin" dependency-type: direct:development update-type: version-update:semver-patch ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * chore(ci): refactor workflows to scope permissions (#1978) * chore(maintenance): rebase conflicts * chore(maintenance): rebase conflicts * chore: rebase conflicts * chore: remove v2 specific release * chore: release patch script * docs: update docs & snippets * chore: address SonarCloud findings * chore: address SonarCloud findings * chore: address SonarCloud findings * tests: coverage * chore: add layers v2 * chore: bump ua version * docs: add install steps to batch * chore: fix install command * chore: remove unused dependency * Update examples/cdk/package.json --------- Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: Erika Yao <[email protected]> Co-authored-by: erikayao93 <[email protected]> Co-authored-by: Ant Stanley <[email protected]> Co-authored-by: Alexander Schueren <[email protected]> Co-authored-by: Sergei Cherniaev <[email protected]> Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* init parser package * add init config * feat(logger): Support for external observability providers (#1511) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter --------- Co-authored-by: erikayao93 <[email protected]> * chore(logger): PowertoolsLogFormatter docstring and variable naming update (#1585) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter * Fixed incorrect return type, renamed variable for consistency * feat(logger): Support for external observability providers (#1511) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter --------- Co-authored-by: erikayao93 <[email protected]> * chore(logger): PowertoolsLogFormatter docstring and variable naming update (#1585) * Updated formatAttributes for additional parameters and LogItem return type * Updated the unit tests to pass with new formatter * Updated Powertool named objects to Powertools * Updated tests to match new naming consistency * Updated for tests for new naming consistency * Updated formatter for new design decisions * Update Logger for ephemeral attributes * Update bringYourOwnFormatter documentation to match new formatter * Fixed incorrect return type, renamed variable for consistency * chore(maintenance): bump dependencies & drop nodejs14x (#1687) * chore: update release script to mark all utilities as alpha * chore: restore version to ease conflicts * chore: release version change * chore: release version change * chore(maintenance): remove `createLogger` and `createTracer` helpers (#1722) * chore(maintenance): bump dependencies & drop nodejs14x (#1687) * chore: add pre-release script * chore: restore deps * chore: added v2 shim * chore(maintenance): remove logger and tracer helper function * chore: remove imports * chore: fix deps & versions * tests: moved unit tests * tests: move logger tests * chore: added v2 shim * chore: added v2 shim * feat(logger): add esmodule support (#1734) * feat(logger): add esm build output * fix(Logger): Remove barrel files update references * test(Logger): update jest/ts-jest to use ESM * chore(Logger): remove unused lodash.merge * fix(logger): reinstate lodash.merge * chore(logger): revert TS assertion * chore(logger): revert format changes * chore(logger): update postbuild to remove incremental tsbuildinfo files * fix(logger): correct reference to types output * feat(logging): add middleware export * chore(logger): replace postbuild script with echo statement * feat(logger): add typesVersions property and barrel files to /types * chore(logger): file not used, can be added back if needed * chore(logger): add space back to README * chore(logger): revert space in README * feat(commons): add esmodule support (#1735) * chore(logger): adapt logger to commons exports * feat(commons): add esmodule support * chore: address sonar findings * chore(commons): exported version * chore: fixed imports in examples * chore(parameters): fixed imports * chore(metrics): fixed imports * chore(tracer): fixed imports * chore(idempotency): fixed imports * chore(commons): test coverage * chore(batch): fix imports * feat(parameters): add esmodule support (#1736) * feat(batch): add esmodule support (#1737) * feat(internal): add esmodule support (#1738) * feat(testing): add esmodule support * chore(all): update imports * feat(metrics): add esmodule support (#1739) * feat(tracer): add esmodule support (#1741) * feat(tracer): add esmodule support * chore(docs): update imports * feat(idempotency): add esmodule support (#1743) * feat(idempotency): add esmodule support * chore(metrics): fix import * chore(ci): v2 release line * chore(ci): fix alpha versioning pre-release * docs(maintenance): add processes tab (#1747) * docs(maintenance): update mkdocs to support tabs * chore(ci): add parallel test npm script * chore(ci): add jest command * docs(maintenance): add testing page to navbar * docs(maintenance): add contributing info * chore: update roadmap * chore: update release drafter workflow to allow for manual trigger * fix formatting * docs: maintainers handbook * chore: link to new location * fix links * Update docs/maintainers.md Co-authored-by: Alexander Schueren <[email protected]> --------- Co-authored-by: Alexander Schueren <[email protected]> * chore(tracer): update warning to better format segment name (#1750) * chore(tracer): update warning in Tracer to better format segment name * chore: linting * chore(internal): remove outdated notice files (#1752) * chore(maintenance): set `removeComments` to` false` in `tsconfig.json` (#1754) * chore(docs): add invisible unicode char to decorator docs (#1755) * chore: remove extra comma * chore(docs): upgrade doc intro * chore(ci): add workflow to publish v2 docs on merge (#1756) * chore(docs): upgrade doc intro * chore(ci): remove mike commands * chore(ci): upgrade mkdocs * feat(logger): align sampling debug logs feature implementation with the other runtimes (#1744) * test(logger): remove logsSampled field, add default sampleRateValue * test(logger): add tests for sampling debug logs feature * feat(logger): change implementation to make sampling decision at per-function level * refactor(logger): remove redundant createLogger method * refactor(logger): remove getSampleRateValue method * test(logger): improve tests * refactor(logger): return createLogger() back with the detailed comment of the method importance * test(logger): add constructor/custom config/env var priority tests for sampling rate feature, improve description * refactor(logger): address review comments * feat(logger): add refreshSampleRateCalculation method and tests * test(logger): adjust end-to-end tests * chore(logger): refactor types and interfaces (#1758) * chore(logger): refactor types and interfaces * chore: grouped type files * chore: fix code smell * chore: fix ci * chore: fix ci * chore(maintenance): bump Middy v4 & run tests (#1760) * chore(parameters): fix esm bundling * chore(parameters): refactor provider constructor to init client as needed (#1757) * chore(parameters): refactor provider constructor to init client as needed * chore(parameters): moved client instrumentation up in baseprovider * chore(parameters): fix code smells * chore(parameters): fix code smells * chore(parameters): change declare client param * chore(commons): update Powertools UA middleware detection (#1762) * chore(commons): fix double ua detection * chore(commons): fix unit test * chore(layers) widen version check in e2e * chore(maintenance): enable `isolatedModules` and isolate cache (#1765) * chore(layers) widen version check in e2e * chore(maintenance): enable isolatedModules * chore: remove redundant comments from tsconfig * chore: changed path of tsbuild cache * fix: idempotency types * chore(idempotency): refactor aws sdk init logic (#1768) * build(tracer): bump aws-xray-sdk-core to latest * build(maintenance): bump aws sdk dev dependencies * chore(logger): set default UTC timezone (#1775) * chore(parameters): add export types * chore(logger): set default utc timezone * chore(logger): pass down envvarsservice to log formatter * feat(parser): add built-in schemas (#1788) * add dynamodb schema * add alb * add parser to v2 build * fix test * add alb * add built-in schema * add more tests for schemas * remove index export * add cloudwatch with base64 zlip transform * add throw test case * formatting * add kafka schema * restructured tests * add vpc lattice and lattice v2 * s3 event notification should extend eventbridge * s3 sqs should extend from sqs * simplify cloudwatch extract from string * keep message as string, instead of empty object * fix detail type of eb and field names * remove duplicated entries * fix homepage URL in readme * improved test coverage * key and value are always present * cleanup unnecessary definitions, widen peerDep version req * Update packages/parser/src/schemas/cloudwatch.ts Co-authored-by: Andrea Amorosi <[email protected]> * clean up events, some fields are imaginary * fix api gw * fix broken IP addresses in examples * add more tests to api gw * fix apigw2 add more tests * add optional scopes to apigwv2 * add optional field back to api gw, stricter methods for vpc lattice * add test for messageId refinement * remove redundant entry * fix sqs * add dmarcPolicy for ses * added tests * moved cw function from kinesis, fix imports * add parser to build step in ci * use any safely here * removed console logs * name, add datetime to strings * narrow string to datetime * refine to url * imports, remove try/catch * add .js extension to imports * moved comment, fixed path * rename event filename to fix events --------- Co-authored-by: Andrea Amorosi <[email protected]> * feat(parser): add schema envelopes (#1815) * first envelope * add abstract class * add tests * add more tests * fix tests * add envelopes * add middy parser * minor schema changes * add more envelopes and tests, refactored utils to autocomplete event files * simplified check * remove middleware from this branch * refactored from class to function envelopes * removed parser tests, should be in another branch * add parser to pre push * consistent naming * feat(parser): implement middy parser middleware (#1823) * add middy middleware * add type to imports * remove schema type, stick with unkown * feat(parser): implement parser decorator (#1831) * feat(parser): add types for built-in schemas (#1838) * add types for built-in schemas * fixed imports * only use top level schema * chore(parser): add parser subpath exports to package.json (#2179) * add exports and type version to package json, including index.js * use index.js as import for coverage * use package lock from main * fix envelope path and add types to exports * use explicit exports instead of * * import type * make export types explicit * adjust imports in tests for coverage, removed unused exports * remove duplicate imports * feat(parser): implement `safeParse` option (#2244) * first draft on safeParse with major refactoring * add safeParse * fixed sns tests * bump coverage * remove throw error and return ParsedResult * remove one level to reduce complexity score * make static methods readonly * simplified cryptic ternary operation into something readble * Update packages/parser/src/parserDecorator.ts Co-authored-by: Andrea Amorosi <[email protected]> * merged * simplify export * add invisible character for decorator rendering * fix docs and tests * Update packages/parser/src/parserDecorator.ts Co-authored-by: Andrea Amorosi <[email protected]> * add comment with description * remove context * remove unintentional safeParse export * add examples to parse standalone function --------- Co-authored-by: Andrea Amorosi <[email protected]> * refresh package lock after merge * docs(parser): add docs for parser utility (#1835) * WIP: parser * fix test imports * remove unnecessary exports * add custom validation * remove unnecessary export * add warning * remove duplicate imports * add types and error handlig * remove comment from annotations * minor changes * revert merge changes * merged package-lock * Update docs/utilities/parser.md Co-authored-by: Andrea Amorosi <[email protected]> * Update docs/utilities/parser.md Co-authored-by: Andrea Amorosi <[email protected]> * adjust imports to new implementation * add safeParse * fixed line highlight * typo * revert index.md, add private scope to snippets packagef * Update docs/utilities/parser.md Co-authored-by: Andrea Amorosi <[email protected]> * add parser to main, fixed zod install command * fix callout indent * fix tooltip --------- Co-authored-by: Andrea Amorosi <[email protected]> * feat(parser): add custom parse error (#2339) * chore: remove rebase leftovers * docs(parser): add utility readme --------- Co-authored-by: Alexander Melnyk <[email protected]> Co-authored-by: Alexander Melnyk <[email protected]> Co-authored-by: Erika Yao <[email protected]> Co-authored-by: erikayao93 <[email protected]> Co-authored-by: Ant Stanley <[email protected]> Co-authored-by: Sergei Cherniaev <[email protected]>

shdq added 6 commits October 13, 2023 16:21

test(logger): remove logsSampled field, add default sampleRateValue

90db1ea

test(logger): add tests for sampling debug logs feature

8b44ac7

feat(logger): change implementation to make sampling decision at per-…

38d0f2a

…function level

refactor(logger): remove redundant createLogger method

11a7995

refactor(logger): remove getSampleRateValue method

62f4063

test(logger): improve tests

8f57cf3

boring-cyborg bot added logger This item relates to the Logger Utility tests PRs that add or change tests labels Oct 13, 2023

pull-request-size bot added the size/L PRs between 100-499 LOC label Oct 13, 2023

dreamorosi assigned shdq Oct 14, 2023

github-actions bot added the feature PRs that introduce new features or minor changes label Oct 14, 2023

dreamorosi reviewed Oct 14, 2023

View reviewed changes

packages/logger/src/Logger.ts Show resolved Hide resolved

refactor(logger): return createLogger() back with the detailed commen…

6e993a5

…t of the method importance

test(logger): add constructor/custom config/env var priority tests fo…

54cf7ea

…r sampling rate feature, improve description

pull-request-size bot added size/XL PRs between 500-999 LOC, often PRs that grown with feedback and removed size/L PRs between 100-499 LOC labels Oct 15, 2023

dreamorosi mentioned this pull request Oct 18, 2023

Maintenance: refactor Logger types and interfaces to improve consistency and clarity #1729

Closed

2 tasks